Implementing Multi-Perspective Context Matching for the SQuAD Task in TensorFlow

The Multi-Perspective Context Matching model introduced by Wang, et al. [1] in 2016 is known to be capable of producing strong results in the SQuAD question answering task. As of this writing, it is tied for 3 place on the SQuAD leaderboard. Implementing the model efficiently is difficult in practice, and the original introduction paper leaves out some implementation details. The goal of this paper is to give practical details regarding my own implementation of this model and its performance, to accompany a code submission that is known to produce reasonably good scores. 1 The SQuAD task This is an introduction to SQuAD. Please see the original paper for more information on the dataset, task, and evaluation. 1 . 1 D a t a s e t The SQuAD question answering task was introduced by Rajpurkar, et al. [2] in 2016 to provide a challenging, large, high quality dataset for evaluating Reading Comprehension (RC). Those authors found existing datasets to be either too small or unreflective of genuine human reading comprehension, and intended for SQuAD to be both large and qualitatively representative of real human understanding. The dataset consists of 100,000+ questions with one or more ground truth answers per question. Each ground truth correct answer is a span of text taken verbatim from the corresponding reading passage the question is associated with. 1 . 2 Ta s k The task is to take pairs of context reading passages and questions, and return the span of text from the context that answers the question. 1 . 3 E v a l u a t i o n Success on the task is evaluated in two ways. F1: This is the average overlap between predictions and correct answers. The F1 is calculated for each prediction/correct answer pair for the answers under each question, the maximum is taken per question, then the average is computed over the per-question max values. 1 SQuAD homepage with leaderboard: https://rajpurkar.github.io/SQuAD-explorer/